1. Field
The present disclosure pertains to methods and apparatus generally relating to the field of sensors used for monitoring space to provide detection, identification and quantification of events or activities. More specifically, the present methods and apparatus utilize opto-electromechanical systems and software to introduce a new field of stand alone object/event detection sensors.
2. Statement of the Problem
A variety of commercially available sensor types may be used to monitor space and report on a wide range of parameters. The sensor types are associated with different technologies, each exhibiting its own set of advantages and disadvantages. There is a need to improve sensor performance, accuracy and the ability to assess parameters of interest within the monitored space.
The most widely used sensors are of a low cost design. These sensors are crude by modern information processing standards. The currently available sensors for motion, for example, are unable to provide any information regarding detected motion and therefore, trigger for any motion that occurs in the field of view, regardless of the source of motion. By way of illustration, when a sensor is installed to turn on lights in a driveway when a car pulls up, the motion detector problematically activates for passing dogs that walk by, trees moving in a breeze, and even for an optical heat gradient created by the rising sun. These latter occurrences are all categorized as false-positive events, since the objective is selective activation for an approaching car. These sensors correctly detect an intrusion so the system is operating with sufficient selectivity to detect an intrusion event; however, specificity is poor because the system cannot discriminate to ascertain the nature of the intrusion. One solution to avoid such problems is to limit the monitored space. For example, one manufacturer of a driveway monitor device states in the product specification that the “signal should be installed at a height of 4 feet to avoid feedback interference errors from the ground.” This solution is less than satisfactory because it fails to accurately detect intrusion events occurring below a certain height because of ground interference.
Current low cost sensors have poor specificity because there is insufficient resolution to discriminate between different objects or events that fall within their field of view. This inability to detect and respond appropriately for a detected object or event is directly responsible for increased cost of support and may even endangers lives. In one example, a small rodent running across the floor can set off a motion detector, triggering an intruder alarm and causing a police response. Some manufacturers offer ‘pet immune’ detectors, so an alarm is claimed to not trigger by the homeowner's dog for home security systems. These systems do not recognize pets. They are quantifying the heat mass of the object using an infrared sensor and, based on some threshold value, signal an alarm accordingly. Obviously, if the dog runs around the house excitedly because it hears the garbage truck outside, the heat mass will elevate possibly triggering an alarm. Furthermore, there is no ability to discriminate between a dog and a small child.
Nor is it the case that higher-end systems solve these problems in a satisfactory manner. The prior art contains a variety of programming techniques related to the processing of digital video images. Considerable prior art surrounds the processing of video images in an effort to recognize objects or events in the monitored or recorded footage. Applications are primarily intended for surveillance or security purposes. These systems use video cameras to capture video footage in the field of view and transmit the video signal, either through a cable interconnect or wirelessly, to a remote processing station where the video footage is processed. These video systems require high data bandwidth and considerable infrastructure to support the transmission of video signals to remote processing stations. These needs greatly increase the cost of deployment and operation. Also, the image analysis is performed post-event at a central processing station, which typically supports multiple video cameras. The system requirement for central processing greatly increases the required processing power of the central station, the complexity of its operation and the process is analyzing history, rather than real-time. The complexity of video systems is further increased to satisfy one of the primary purposes, which is preservation of records for future reference. Furthermore, video cameras are expensive and complex devices, which lower their reliability.
Digital imaging systems have become very inexpensive to produce. As a result, image creation is a standard feature on most modern handheld communication devices, e.g., cell phones and PDAs. Image creation technology has been commoditized to greatly reduce the cost to create a digital image. These commodity components provide a low cost, stand-alone, reliable image capture hardware platform.
Recently, systems and methods have been disclosed that have taken advantage of the commoditization of image creation technology and incorporated imaging capability into monitoring applications. U.S. Pat. No. 7,231,654, entitled “Remote Monitoring Method and Monitor Control Server” (“the '654 patent”) describes method where an image capture device is added to a remote monitoring device that is connected to monitor control server. The '654 patent essentially adds an image capture capability to a conventional motion detector used for security monitoring, where an image is captured only when the motion detector triggers and the image is transmitted to the control server without any image processing being performed by the remote monitor. The control server then formats the image for transmission to a mobile terminal, such as a PDA or cell phone, where the image may be viewed by a user.
The '654 patent repeats the problems of the prior sensor art. The disclosure does not provide any imaging processing at the remote monitor and, in fact, does not provide any image processing throughout the entire system. The image capture capability is only used to capture an image and pass it through the system using various communication methods to enable the image to be reviewed remotely. The '654 patent does not provide any improvement to the sensor art and only offers improvement to the notification methods for existing security systems.
In another example of prior art that falls short of providing any improvement to the general sensor art is disclosed in U.S. Pat. No. 6,697,104 (“the '104 patent”), entitled “Video Based System and Method for Detecting and Counting Persons Traversing an Area Being Monitored.” The '104 patent discloses a dedicated remote video device having a single function, namely, to count the number of people that enter or exit a monitored area. The data collected by the remote monitoring device must be uploaded to a remote processing station where additional processing activity is required to provide any useful value. Several prerequisites exist regarding the deployment environment that severely limits the functional value of the disclosure. The '104 patent provides some limited ability to ‘configure’ the remote monitoring device once it is installed at its deployment location to account for the environment of use however, the configuration capability is limited to the ‘area of interest,’ subdividing the field of view. Accordingly, the ‘counting’ function is limited to only a part of the captured video signal.
Using video signals, the remote monitoring device of the '104 patent processes frames of video but never actually recognizes the presence of a person. There is merely recognition of artifacts of “what might be a person.” For example, if a dog were to pass through the field of view, the system may or may not count the dog as person. This happens because the '104 patent relies upon on a static background image. There is subtraction of this background to identify a patterned characteristic change in the background that is assumed to represent a person; however, this is not a process that recognizes the object itself. Only an artifact of the image is observed as a change in the background. Furthermore, the '104 patent registers person counts in some internal counters that must be uploaded to another computer system where the counts must be processed and formulated into a useable format for a subscription based customer model. The remote monitoring device does not produce output that contains any actionable information. The single, dedicated function of the remote monitoring device cannot be reprogrammed, and its inability to receive ‘count’ objects other than people is limiting.
U.S. Pat. No. 7,190,259 (“the '259 patent”) discloses a lens arrangement to create a 360 degree view for a video imaging component. The system displays the captured image on a screen. The lens is deployed in the exterior rearview mirrors of an automobile. The image is displayed for driver viewing. The omni-directional vision sensor merely delivers output for presentation of the image and does not process the image.
U.S. Pat. No. 7,414,647 (“the '647 patent”) describes another lens arrangement to create a 360 degree view for a video imaging component. Again the system displays the image on a remote terminal screen. The image capture and processing device has limited processing to format the image data into either panoramic or perspective views for display purposes. A mobile body detecting section is disclosed for detecting a moving body in the field of view. The primary application disclosed is for deployment in an ATM to capture images of ATM users. A ‘communication section’ telecommunicates with an external terminal device. The data can be transmitted either wirelessly or via cable interconnect but again, its limited strictly to image data. There is no discussion anywhere in the disclosure of the image capture and processing device receiving any information or of the communication section being used to receive data.
U.S. Pat. No. 7,200,246 (“the '246 patent”) discloses an imaging system and method directed toward industrial equipment safety by monitoring an ‘area of interest’ and detecting an object entering the area. The image capture and processing device contains image processing algorithms for object detection, an output signal and the ability to connect a PC to the image capture and processing device for configuration. However, the disclosed capability all relates to an ‘area of interest’ in the manner of a motion detector and do not address the selective identification of ‘objects of interest’, nor are any image processing functions or algorithms transferred to image capture and processing device through the PC connection.
The ‘area of interest’ is monitored by the image capture and processing device employing a defined border area. The image processing relies on the use of a reference image, where sampled images are subtracted from the reference image and changes in pixel pigmentation are used to determine if an object has ‘breached’ the border of the monitored area of interest. While the disclosure includes discussion of establishing a ‘threshold’ for the number of pixels required to trigger an output signal, even referencing a “hand”, there is no attempt by the '246 patent disclosure to recognize the object breaching the border.